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CONTROLLING PROCESSOR -BASED 
SYSTEMS USING A DIGITAL CAMERA 

Background 

This invention relates generally to processor-based 
systems and particularly to such systems which have a 
digital camera coupled to the system. 
5 A variety of processor-based systems come with a 

digital camera or are adaptable to be coupled to a digital 
camera. Commonly, digital cameras are tethered through an 
appropriate input port to personal computers. A number of 
users operate the digital cameras continuously through a 

10 tethered connection to the computer. 

The camera may be maintained always "on" , ready to 
provide a useful service as the opportunity arises. The 
camera may be used to implement a video conferencing 
feature or to stream video over the Internet as examples. 

15 A number of techniques are available for controlling 

processor-based systems. Processor-based systems may 
receive input commands using a mouse which is tethered to 
the computer, using an infrared mouse which controls the 
computer remotely, using a variety of remote control 

2 0 devices, and using keyboards either tethered to the 

computer or keyboards that are coupled to the computer 
through infrared or other airwave communications, as a few 
examples . 
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In addition, a number of computer systems transition 
between power consumption states in response to periods of 
sustained user inactivity. That is, if the keyboard, for 
example, is not operated for a given time period, the 
5 system may transition to a lower power consumption state. 
With ensuing periods of inactivity, the system may 
progressively transition to even lower power consumption 
states. In this way, the amount of power that the system 
consumes may be controlled. In addition to environmental 

10 advantages, the lifetime of the computer may be enhanced by 
such powering down in response to inactivity. 

Sensors may detect the user's presence in proximity to 
the keyboard. These sensors may be piezoelectric sensors 
that detect when the user is poised over the keyboard, for 

15 example. Again, this type of sensor provides additional 

information to the computer system to make a judgment about 
whether or not to transition to lower power consumption 
modes. If the user is poised over the keyboard about to 
operate the keyboard, it makes no sense to transition to a 

20 lower power consumption mode only to immediately transition 
back to a higher power consumption mode. Not only is this 
awkward, but needless transitions waste time and system 
resources . 

Thus, there is a continuing need for better ways to 
25 control the power consumption of computer systems. 
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Summary 

In accordance with one aspect, a method of controlling 
a processor-based system includes receiving video 
information from a camera. The power consumption state of 
5 the system is controlled based on the video information. 
Other aspects are set forth in the accompanying 
detailed description and claims. 

Brief Description of the Drawings 
Fig. 1 is a front elevational view of a processor- 
10 based system in accordance with one embodiment of the 
present invention; 

Fig. 2 is a flow chart for software for implementing 
one aspect of one embodiment of the present invention; 

Fig. 3 is a flow chart for software for implementing 
15 another aspect of one embodiment of the present invention; 

Fig. 4 is a flow chart for software for implementing 
another aspect of one embodiment of the present invention; 

Fig. 5 is a flow chart for software that implements 
another aspect of one embodiment of the present invention; 
2 0 and 

Fig. 6 is a block diagram for the embodiment shown in 
Fig. 1. 

Detailed Description 
A processor-based system 10 shown in Fig. 1, includes 
25 a processor-based housing 12, a keyboard 13 , a display 14 
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and a digital camera 16 which may be coupled by a tether 
(not shown) to the housing 12. As illustrated, the camera 
16 is positioned to observe the area in front of the 
processor-based system 10. Thus, the camera 16 detects the 
5 user's presence as well as the lighting conditions 
proximate to the system 10. 

Referring now to Fig. 2, light monitoring software 18 
stored on the system 10, is responsible for controlling the 
power consumption state of the processor-based system 10 in 

10 response to ambient light. A timer is reset (block 20) and 
the keyboard and mouse are checked for their current 
condition (block 22) . At diamond 24, a check determines 
whether the keyboard and mouse have just been used. If so, 
the timer is again reset. If not, the check at diamond 2 6 

15 determines whether the timer has expired. If the timer has 
expired, indicating a period of inactivity exceeding a 
predetermined time period, a video frame is grabbed as 
indicated in block 28. In other words, the camera 16 
provides a video frame which may be analyzed at block 30. 

2 0 In particular, the luminance value of the frame may be 
computed at block 30. 

A subsampling of the pixels of a frame may be 
analyzed, for example, approximately 250 pixels in one 
embodiment of the invention, evenly distributed throughout 

25 the frame. Each pixel's luminance value is computed by 

converting its red, green, blue (RGB) color space value to 
a hue, saturation and luminance (HSL) color space value 
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using well known techniques. Luminance is the brightness 
portion of a composite video signal. The pixel luminance 
values are simply summed and divided by the number of 
samples. The result is the return luminance value. 
5 Referring to Fig. 3, the software 42 for determining 

the return luminance value begins by taking the video frame 
as described previously (see block 44) . The pixel stepping 
for the pre-set sample set, approximately 250 pixel samples 
in this example, is computed (block 4 6) . The software 

10 steps to the next pixel location (block 48) . The pixel at 
the next location has its RGB color space values converted 
to an HSL format (blocks 5 0 and 52) . The luminance value 
of the next pixel (from the HSL space) is added to the 
accumulated luminance total value, as indicated in block 

15 54. If there are more pixels to complete the sample set, 
the flow iterates. If not, the luminance total is divided 
by the number of pixel samples as indicated in block 58. 
The result is then returned to the flow in Fig. 2, as 
indicated at block 60. 

2 0 Returning to Fig. 2, next the motion value is computed 

(block 32) . As shown in Fig. 4, software 62 for computing 
the return motion value begins by taking the video frames 
from the previous steps (Fig. 2) as indicated at block 64. 
The pixel stepping for a pre-set number of pixels, for 

25 example approximately 250 pixels, is computed, as indicated 
in block 66. The flow steps through each successive pixel 
as indicated in block 68. At block 70, the pixel at the 
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new location in the old frame and new frame are accessed. 
The red value in the old frame is subtracted from the red 
value in the new frame and this process is repeated for the 
blue and green values as indicated in block 72 . An 
5 absolute value of the difference is computed (block 74) . 

If the result is greater than 50 or some other noise 
threshold (diamond 76) , the result is added to the total 
motion return value (block 78) . At diamond 80 a 
determination is made as to whether there are additional 

10 pixels in the initial set of 250 pixel samples. If not, 
the motion value is returned (block 82) . Otherwise, the 
flow continues to iterate until all the pixels in the 
sample have been processed. 

Returning again to Fig. 2, at diamond 34 the flow 

15 determines whether the returned luminance value differs by 
more than 40 percent. Of course, 40 percent is merely an 
exemplary threshold for testing the returned luminance 
value. If so, a check at diamond 3 6 determines whether 
this is the fourth time (or some other number) in a row 

20 that the luminance value remained at this level. If so, a 
check at diamond 3 8 determines whether motion has occurred 
during the time period. If not, appropriate changes can be 
made as indicated in block 40. Among the changes that may 
be made are to activate a screen saver, implement a power 

2 5 management decision or implement a system utility. 

Once the camera 16 is activated, it may capture a 
frame every second in one embodiment of the invention. The 
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system 10 computes the luminance value for each frame and 
compares its value to the luminance value computed for a 
previous frame. If the number is significantly lower than 
that of previous frames, the software starts to suspect the 
5 lights might be out. It keeps capturing frames, computing 
both a luminance value and a motion value. If the 
luminance value continues to be low for several frames and 
the motion value is also low, the computer screen saver and 
power management mode may be activated. When full power 
10 management is turned on, a keyboard or mouse input signal 
may be used to reactivate the computer. However, the 
camera 16 can continue to run with the monitors, printers 
and hard drives powered down, as long as the processor 
remains on. 

15 The camera continues to capture frames while the 

computer is in the lower power usage mode or screen saver 
mode. It compares the frames and computes the luminance 
value for each frame. If the luminance suddenly increases, 
the system is immediately returned to full power status 

2 0 and/or the screen saver is deactivated. If the luminance 

values do not change significantly, and motion occurs for a 
time period (for example continuously for three seconds) , 
the system may be returned to full power status and/or the 
screen saver may be deactivated. 

2 5 Turning now to Fig. 5, the software 84 for leaving the 

screen saver, power management, or system utility modes 
begins by checking the keyboard and mouse status (block 
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86) . At diamond 88, a check determines whether the 
keyboard or mouse was just used. If so, the computer is 
returned to full operational status as indicated in block 
102. The timer is reset, as indicated in block 104, and 
5 the keyboard and mouse states are checked (block 106) . 

At diamond 108, a check determines whether the 
keyboard or mouse were just used. If not, the flow 
determines (at diamond 110) whether the timer has expired. 
If so, the screen saver, power management mode or system 

10 utilities may be reactivated, as indicated in block 112. 

Otherwise, the system continues to check the keyboard and 
mouse state, as indicated in block 106. 

If the keyboard and mouse were not used as determined 
at diamond 88, a video frame is grabbed as shown in block 

15 90. A luminance value and motion value are computed as 

indicated in blocks 92 and 94. If the luminance value does 
not differ by a preset amount (for example more than 4 0% at 
diamond 96) , a check at diamond 98 determines whether 
motion has occurred. If so, a check at diamond 10 0 

2 0 determines whether motion has occurred two times in a row. 
If so, the system returns to full activation (block 102) . 
If not, the flow returns to block 86 and checks keyboard 
and mouse states. 

Referring now to Fig. 6, the system 10 may include a 

25 processor 114 coupled to an interface 116. The interface 
116 may be a chipset or bridge, as two examples. The 
interface 116 may be coupled system memory 118 and a 
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display controller 122. The display controller is coupled 
to the display 14. 

The interface 116 may also couple a bus 126. The bus 
12 6 in turn may be coupled through an interface 128 to the 
5 camera 16. In addition, the bus 126 may couple an 

interface 132. The interface 132 may be coupled to a bus 
136 and to a storage device such as a hard disk drive 134. 
The software 18, 42, 62 and 84 may be stored on the hard 
disk drive 134 . 

10 The bus 136 is coupled to conventional components such 

as a serial input/output device 138. The device 138 
couples a mouse 142 and a keyboard 13 . The basic 
input/output system (BIOS) 144 may also be provided on the 
bus 136. 

15 While the present invention has been described with 

respect to a limited number of embodiments, those skilled 
in the art will appreciate numerous modifications and 
variations therefrom. It is intended that the appended 
claims cover all such modifications and variations as fall 

20 within the true spirit and scope of this present invention. 
What is claimed is: 
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1 1. A method of controlling a processor-based system 

2 comprising : 

3 receiving video information from a camera; 

4 analyzing said information; and 

5 controlling the power consumption state of said 

6 system based on said video information. 

1 2 . The method of claim 1 further controlling the 

2 operation of system utilities based on said video 

3 information. 

1 3 . The method of claim 1 further controlling the 

2 operation of a screen saver based on said information. 

1 4. The method of claim 1 wherein analyzing said 

2 information includes calculating a luminance value from 

3 said video information. 

1 5 . The method of claim 4 including determining 

2 whether said luminance value has changed by a given amount. 

1 6 . The method of claim 5 including determining 

2 whether the luminance value has changed by a given amount 

3 for a given time period. 
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1 7. The method of claim 4 including determining a 

2 value indicative of whether the video information indicates 

3 motion. 

1 8 . The method of claim 7 including determining 

2 whether said motion has persisted for a given period of 

3 time. 

1 9. The method of claim 8 including using said value 

2 to control the power consumption state of said system. 

1 10. An article comprising a medium for storing 

2 instructions that cause a processor-based system to: 

3 analyze video information; and 

4 control the power consumption state of said 

5 system based on said video information. 

1 11. The article of claim 10 further storing 

2 instructions that cause a processor-based system to control 

3 the operation of system utilities based on said video 

4 information. 

1 12. The article of claim 10 further storing 

2 instructions that cause a processor-based system to control 

3 the operation of a screen saver based on said information. 
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1 13. The article of claim 10 further storing 

2 instructions that cause a processor-based system to 

3 calculate a luminance value from said video information and 

4 to use said luminance value to control the power 

5 consumption state of said system. 

1 14. The article of claim 13 further storing 

2 instructions that cause a processor-based system to 

3 determine whether said luminance value has changed by a 

4 given amount . 

1 15. The article of claim 14 further storing 

2 instructions that cause a processor-based system to 

3 determine whether the luminance value has changed by a 

4 given amount for a given time period. 

1 16. The article of claim 14 further storing 

2 instructions that cause a processor-based system to 

3 determine a value indicative of whether the video 

4 information indicates motion. 

1 17. The article of claim 16 further storing 

2 instructions that cause a processor-based system to 

3 determine whether said motion has persisted for a given 

4 period of time. 



12 



1 18. The article of claim 17 further storing 

2 instructions that cause a processor-based system to use 

3 said motion information to control the power consumption 

4 state of said system. 

1 19. The article of claim 10 further storing 

2 instructions that cause a processor-based system to 

3 determine whether the lights are on proximate to the 

4 system. 

1 20. A method of controlling a processor-based system 

2 comprising: 

3 receiving video information; 

4 analyzing said information to develop luminance 

5 information; and 

6 controlling the operation of software on said 

7 system based on said luminance information. 

1 21. The method of claim 2 0 further including 

2 controlling the power consumption state of said system 

3 based on said luminance information. 

1 22. The method of claim 20 further including 

2 controlling the operation of a screen saver based on said 

3 luminance information. 
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1 23. The method of claim 20 further including 

2 controlling the operation of system utilities based on said 

3 luminance information. 

1 24. The method of claim 24 including determining 

2 whether the video information indicates motion. 

1 25. An article comprising a medium for storing 

2 instructions that cause a processor-based system to: 

3 analyze video information to develop luminance 

4 information; and 

5 control the operation of software on said system 

6 based on said luminance information. 

1 26. The article of claim 25 further storing 

2 instructions that cause a processor-based system to control 

3 one or more of the power consumption state of said system, 

4 a screen saver, or system utilities, based on said 

5 luminance information . 

1 27. The article of claim 26 further storing 

2 instructions that cause a processor-based system to 

3 determine whether the video information indicates motion. 
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1 28. A processor-based system comprising: 

2 a processor, said processor coupled to a storage 

3 device; 

4 a digital camera coupled to said processor; and 

5 said storage device storing software that 

6 controls the power consumption state of said system based 

7 on information received from said camera. 

1 29. The system of claim 28 wherein said software 

2 controls the power consumption state of said system based 

3 on information from said camera indicative of motion 

4 proximate to said camera. 

1 30. The system of claim 28 wherein said software 

2 controls the operation of system utilities based on 

3 information from said digital camera. 
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CONTROLLING PROCESSOR -BASED 
SYSTEMS USING A DIGITAL CAMERA 

Abstract of the Disclosure 
A processor-based system may be controlled using a 
digital camera. The digital camera can provide luminance 
5 and motion information which may be analyzed to determine 
whether to alter one or more of the power consumption 
state, the operation of system utilities, or the operation 
of a screen saver. 
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